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ABSTRACT 

This paper is an appraisal of the Colorado 
Elementary Science Project (CESP) , a state-wide program to provide 
inservice education and assist school districts in the implementation 
of new elementary school science curricular programs. The University 
of Colorado and the Colorado State Department of Education cooperated 
in assisting school districts in their initial introduction of AAAS 
Science - A Process Approach, Elementary Science Study, and Science 
Curriculum Improvement Study. Selected elementary teachers were 
trained in the use of a new curricular program over a period of one 
school year while they were using the new materials with their own 
students. This was followed by a two-week summer session in which 
they were prepared to teach other teachers. Results of the evaluation 
indicate that: (1) one semester inservice course produced a change in 

the teacher* s style of teaching science, (2) the major constraint 
limiting the full implementation of the new elementary science 
programs was the quantity of equipment and materials needed, and (3) 
principals became involved in implementation only if specific efforts 
were made to get them involved. (BR) 
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The evaluation reported herein is an appraisal of the Colorado 
Elementary Science Project (CESP)^, as a statewide program to pro- 
vide ^nservice education and assist school districts in the implemen- 
tation of new elementary school science curricular programs. As 
such^ this report is not research in the sense of providing generalized 
knowledge about human behavior. The information provided^ however, 
is of more than local interest since it should be useful to persons 
attempting to provide large scale inservice education as a means of 
implementing curricular change. 

The CESP was developed x-/ith the intent of assisting all school 
districts in the state of Colorado which desired assistance in their 
initial introduction of the ESS, SAPA, or SCIS^ science prograius. 

Since the personnel are limited almost entirely to members of the 
sponsoring agencies, the University of Colorado and the Colorado State 
Department of jzJducabion, the project must of necessity employ a 
multiplier principle to reach the large number of elementary teachers 
needing assistance. Selected elementary teachers are trained in 



This project is now operating under the third year of NSP 
funding through the Cooperative College-School Science Program. 

2 

ESS - Elementary Science Study , a program of the Education 
Department Center; SAPA - Science --A P rocess Approach , developed 
under ^ sponsorship of the Am.erican Association for the Advancement 
of Science; and SCIS - Sc ience Curric ulum ICmprovement Study, a 
program developed by the organization of the same nameli ^ 



the use of a new curricular program over a period of one school year 
while they are using the new materials with their own students. 

This is followed by a two week summer session in which they are pre- 
pared to teach other teachers. They then undertake almost the en- 
tire responsibility for teaching a three semester hour course which 

is a key part of implementing a new curricular program in a given 
school district*^ 

Participation in the project requires considerable local 
school district responsibility. Inservice classes are established 
only in districts where the necessary student equipment is provided, 
the initial selection of participants is done locally, and after 
the initial 16 months under CESP sponsorship, continued implementa- 
tion of the new programs is a local district matter. The focus of 
the project is to aid school districts in beginning the implementa- 
tion of new curricular programs, not simply to provide inservice 
education for teachers. 

A major facet of the CESP involvement with a school district 
is the instruction of large numbers of teachers which is provided 
by the selected teachers who were trained during the proceeding 
year. This instruction is provided within the fram.ework of a three 
semester hour course (undergraduate University credit) conducted 
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weekly after regular school hours and visits by the instructor to the 
classrooms of the teachers taking the class. Extensive work by the 
teachers with the new materials and with groups of children using 
the materials are part of the course. The observation and analysis 
of teaching, both live and videotaped, are employed. 

SELECTED QUESTIONS 

That portion of the evaluation of the CESP reported herein 
is limited to the study of selected questions which are likely to 
be of interest to persons making decisions in the operation of 
similar types of programs. These questions are given below along 
with information that bears upon their answers within the CESP as 
presently operated. Because they xi^ere selected on the basis of 
their importance as well as the information available to answer 
them, there are several for which only tentative and limited in- 
formation is presently available. 

1* Does the teachers * style of teaching change as a result 
of their participation in the pro ject ? This was one of the first 
questions to which major attention was given in the evaluation. 

The most important outcome of the project would be changes in what 
children and teachers do in the classroom. Since the project 
operates on a multiplier principle to reach large numbers of 
teachers, the evaluation of classroom changes had to be conducted 
in the classrooms of those teachers who were being taught by other 
teachers if the true impact of the project was to be assessed. For 
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this reason an evaluation of change in teaching style was conducted 
in the classes of teachers who were randomly selected from among the 
approximately 150 teachers who were members of ESS or SAPA classes 
in the three original districts in the project. 

Twenty-eight teachers were selected randomly as a group from 
which pre-test data could be acquired, A different 28 were selected 
randomly from the l50 teachers as the source of post-test data. 

In Campbell-Stanley notation this could be represented as follows: 

R 0 X 
R X 0 

This qua si- experimental design provides pre and post data without 
introducing the possibility of a testing effect and a pretest- 
trcatniGnt intorciction effect as influences on the posttest data. 

History and saturation , of course, are still threats to internal 
validity but are judged to be less serious problems than the former 
two threats to internal validity. 

Assessment of changes in teaching style was accomplished 
through observation of teaching and the use of the Teaching Strate- 
gies Observation Differential (TSOD)> This instrument provides 
an overall rating of the style of teaching employed by a teacher, 
on a continuum on which the extreme scores are 1 and 8, The ex- 
tremes, termed expository-direct and inductive- indirect respectively. 



Torn Andepon, Ronald D., Joseph A, Struthers, and Helen H. 

? Verbal and Non-Verbal Observation Instru- 
ment, a paper presented at the annual convention of the American 
1970^^^°^^^ ^®®®^^®^ Association, Minneapolis, Minnesota, March 3, 
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are represented by the following diagrams 
(S)4^ (T) (E) 



(S) 4 (T) 




expository-direct 



ind ir e c t Iv e - ind ire c t 



S represents student, T represents teacher, and E represents the 
classroom environment including the learning materials employed and 
the observable science phenomena under study. In the expository- 
direct style, the teacher acts as interpreter of natural phenomena 
for the children and is the "filter" through which all information 
is dispeaeed. In the other style, children interact directly with 
science materials and formulate their own conclusions to student 
generated questions; the teacher's role is largely that of facili- 
tator and supplier of materials. 

Since elementary science programs such as ESS, SAPA, and 
SCIS are focused upon student use of materials and student inquiry, 
the inductive-indirect mode represents the pole toward which the 
inservice classes were aimed, A one-tailed test of change in style 
for the pre and post groups was planned because of this anticipated 
di]?ection of change. 

Because a teacher's style of teaching would be influenced 
by the subject matter being studied and the materials used, all 
teachers were provided with the same materials for use during the 
class scheduled for acquiring observations of teaching, A small 
unit on earthworms was written I'^hich included various suggestions 
from which the teacher could make her own selections for that 
day. All teachers were supplied with the same materials, including 



earthworins, in sulTicient quantities for all children to be working 
with them. The written material provided for the teacher contained 
suggestions of uses of the materials that ranged from expository- 
direct to inductive -indirect, 

A 20-30 minute segment of teaching was recorded with a port- 
able video tape recorder as a pre-test in September prior to the 
inservice classes for the teachers* In January^ after completion 
of the inscrvice classes, a similar sample of teaching was recorded 
as a post-test for the other group of randomly selected teachers. 

By recording the samples of teaching and having both the pre- and 
post-measures rated at a later date by raters who did not know to 

which group each teacher belonged, possible rate:-.? bias was eli- 
minated. 

The rating of the samples was done by a group of four raters, 
with two raters rating each sample. The raters followed a rotat- 
ing pat-cern which resulted in each of the six possible combinations 
of two raters rating approximately equal number of samples* The 
Hoyt inter-rater reliability^ ranged from .89 to .97 with an average 
inter-rater retrobility of o9lj... 

The results of the ratings are given below: 



s im 



Winer, 
(New York: 



J, Stat is tical Procedures in 
McGraw-Hill), 1962,' pp* 126-128'. 
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pre 


post 


mean 






standard deviation 


lo06 


,89 


n 


26 


,28 



This gain of approximately 2/3 of a standard deviation is 
significant at the oOl level with a one tailed testo It is concluded 
that the one semester inservice course produced a change.. in the 
teacher's style of teaching scienceo 

2, ^ what extent do the teachers actually i^_e the new 

— materl^]^? Even though the teacher's style of teaching 
changes in the desired direction does not insure that they actually 
use the new science progranu. The managerial problems that accompany 
a program in which the majoi student activity is Individual or 
small group work with "hardware,” rather than, reading and recita- 
tion as an entire class, are sufficient to be a significant deter.rent 
to many teachers’ use of the program. At this writing only limited 
hard data" is available to use in ascertaining the extent to which 
the materials are in usee A follow-up survey which will provide 
such information ..'or teachers who completed the inservice course 
over one year ago is now in the planning stage. 

Appendix A contains a thirty item Likert type questionnaire 
and the responses of ij .13 teachers who were attending the last 
session of the inservice classes held during the second year of 
the projects In the sections entitled "the elementary school 




«• 0X> 

science program” and ”your classroom" are several items that give 
hints about what a teacher can be expected to do once she finishes 
the inservice program. These responses lead the writers to conclude 
that most of the teachers will continue using the new materials 
regularly in the future the acquisition of materials and equip- 
ment is not a problem. The item which was expected to give the 
most direct information on this question, number 23 ^ is not too 
informative because it asks about extensive use and during this time 
many teachers were still having difficulty obtaining some of the 
materials as indicated in number 30 * A more complete answer to the 
question must await the follow-up, 

3o What the major constraint limiting the full Implement a - 
new elementary science programs ? Conversations with 
project staff members, school administrators and teachers indicates 
clearly that ^ major constraint is the quantity of equipment 
and materials needed. The extensive use of materials and equipment 
by individual chi3,dren is both the major strength and weakness of 
programs such as ESS, SAPA, and SC IS, Although the cost of the 
materials is certainly a factor, money alone is not the solution. 

The logistics of having the needed materials available for the 
teacher at the proper time is an enormous task. It would be diffi- 
cult to overstate the importance of attacking this problem in any 
effort to implement a program such as ESS, SAPA, or SCIS. Based 
on the Project evaluation the following recommendations are 



offered: 



Inservice classes should not be held unless adequate- 
supplies of materials are available to enable all parti- 
cipating teachers to use the new program with their stu- 
dents. 

The ordering of all equipment should be completed as 
early as possible. 

Each district must establ ish a system of distribution 
and determine which supply problems are to be handled 
by the individual schools and which are to be handled 
by the central administration. 

An orientation session on equipment and accompanying 
problems should be provided for principals. This can 
be part of the informational sessions on the new science 
programs which should be provided for administrators. 

Storage space and shelves must be provided in each 
school. 

A system of inventorying, check-out, and reordering must 
be establishedo 

It is often advantageous for one teacher to have major 
responsibility for any centrally maintained equipment 

in a school and for the ordering and distribution of ex- 
pendable materials , 

It is often feasible to appoint otudenc associates 
(probably two or more 5th and 6th graders) for a school 
to assist the teachers and principal. 



i. A petty cash fund should be available for teachers to 

draw upon for inexpensive items that are needed on short 
notice • 



if.. We^ th^ £^ t_Qpipals of the participating schools opti <- 
l ijyolY i d ? visits were made to the principals of representa- 
tive schools for the purpose of acquiring their opinions of the new 
science programs and the efforts being made to implement them, as 
well as to ascertain the extent of their involvement in these 
efforts and their support of them. Principals were often only 
marginally involved if specific efforts had not been made to get 
them involved. The principal's central role in solving the equip- 
ment problems mentioned above is but one illustration of the im- 
portance of having the support of a knowledgeable principal. 



5. Do district s cpnt_inue implementation work on 
their own? The Colorado Elementary Science Project operates on the 
assumption that if an adequate number of teachers are trained to 
as inservice instructors, they have the experience of teach- 
ing a class once under supervision, and a sizeable percentage of 
teachers in the district have a successful experience with the 
new program during their initial exposure to it, the dictrlct will 



be willing and able to continue the implementation on their own. 

A key measure of the success of the project is the extent to which 
the effort is continued. A systematic assessment of the degree to 
which this has been done and the reasons for action or lack of 
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action is yet to be aoooBiplished, Informal communication viith 
the districts involved, however, indicates that several dis- 
tricts are establishing extensive programs for this purpose. 

6. Is University credit .justifiable for the thre e 
s emester hour course taugh t ^ other teachers ? This question 
has been raised on several occasions by the University Ad- 
ministration. In many ways the answer is a matter of value 
judgements; it is difficult to answer in any other way than to 
examine the means of quality control employed. The three main 
factors upon which quality control depends are: (1) selection, 

(2) extensive preparation, and (3) supervision of the teachers 
who serve as instructors of fellow teachers. 

The teachers who enter the initial phases of the program 
are selected by the participating districts. Additional selec- 
tion is done by the CESP staff. For example, during the first 
year of the project, approximately one-third of the teachers 
entered the program ’■/ere selected as instruc.tors. The pre- 
paration for teaching other teachers incl’udes one full year of 
using the new sciaice program with the children in their own 
class while concurrently enrolled in inservice classes total- 
ing approxiira tely 5 semester hours. This is followed by a two- 
time session in the summer that is designed specif i- 
y to prepare them to teach other teachers. The supervision 
is provided by a member of the CESP who devotes full time to the 
supervision of the classes being taught by these teachers. 
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The credit question becomes more involved when related 
to the objective of having each district continue its implemen- 
tation efforts on its ownj under these conditions the supervi- 
sion discussed above is not presento It is possible for one of 
the instructors who has been trained through the project to 
teach the inservice class for credit under the University's 
Extension Division independent of the CESP if he has a master's 
degree and a substantial number of hours in science. Since 
only about of these elementary teachers have master's 
degrees, further implementation efforts by the district are 
greatly curtailed if credit is important. Since credit is an 
important incentive (and for other reasons), a new approach is 
being employed this year- Teams consisting of a secondary 
science teacher who has a master's degree and an elementary 
teacher are being prepared as instructors for the fall of 1970* 
This team should meet the formal University requirements and in- 
crease the possibilities of continued implementation efforts 

by the local districts after their participation in the Colorado 
Elementary Science Project. 



-13- 



APIENDIX A 

The items below are given as they appeared on an anonymous 
questionnaire administered to 4^3 participants in the final 
phase of the second year of the CESP. The meaning of the symbols 
is as follows: SA = strongly agree, A = agree, ? = undecided, 

D = disagree, and SD = strongly disagree. The number ical values 
given are the percentage of persons checking each response. The 
difference between the total of the five percentages for each 
item and 100^ is the percentage who did not respond to the item. 
The percentage of nonresponse varied from .7^ to 2,1^, 



elementary school science program 

(This section pertains to the ESS, AAAS, or SC IS .program, 

whichever it is you are using,) 







SA 


A 


9 

• 


D 


SD 


1. 


This elementary school science 

program will better prepare pupils 
for junior high school. 


28,3 


5k^o 


13.6 


1.9 


.7 


2. 


In this science program too much 
time is spent ’’messing around,” 


k.k 


13.3 


11.1 


50.6 


l8, 6 


3. 


The source of answers to questions 
is too often the laboratory 
rather than the teacher. 


8.2 


vO 

• 

1 — 1 


i.r\ 

• 

1 — 1 


37.0 


• 

GO 

1 — 1 


4. 


The materials are realistic for 
use by all children. 


30.5 


k5.5 


8.2 


GO 

c 

1 — 1 
1 — 1 


2.4 


5. 


The materials are realistic for 
use by all teachers. 


21,0 


k5.o 


12.3 


17.2 


3.6 


6, 


The daily preparation required to 
use those materials is reason- 
able . 


17.0 


55.k 


10.6 


1 1 

O 

OJ 
1 — 1 


3.6 




V 



1 





7 . 



8 . 



The results of the program justi- 
fy the effort required in their 
use, 

I can recommend the materials to 
my colleagues. 





A 




D 




26,2 


47. a 


1 — 1 

0 

CM 




14 


28.6 


53.3 


9.9 


5.6 


1.4 



-The inservice education program 



9 » Too much time was given to demon- 
stration teaching, 

10 , Too much time was spent by the 
instructors observing my class- 
room teaching. 

11 , Too much time was spend working 
through the lab activities of 
the elementary school science 
program. 



7.5 


14.5 


3,2 4.8.9 20.1 


1.4 


.7 


6.0 43.3 47.7 


4.8 


7.8 


13.1 47.5 25.2 



12 , More time should be spent pre- 
senting science background, 

13 * More^time should be spent on the 
specific activities which will 
be used in our own classes, 

14* Too much time was spent telling 
us what constitutes good science 
teaching, 

1?. Participants should have been 
instructed in groups divided 
according to the grade level 
they teach, 

16, More attention should have been 
given to continuity of the 
materials in the elementary 
school science program, 

17 • The instructors were competent 
and gave us sufficient personal 
attention. 



6.8 22.3 16,7 37.3 15.5 
17.9 38.3 14.0 23.2 5o8 



3.2 6.5 8.7 53,3 26,9 

18.4 25,4 13.6 31. s 9.7 

12.6 34,9 19.8 28.1 2.9 

32.7 43.8 10.6 8,7 3,2 



i 



- 15 - 



l8. The films were of little educa- 
tional value o 

19» The principal should determine 
the science program (ESS, AAAS, 
SCIS) used in his building, 

20. The consultant or supervisor 
should select participants for 
inservice programs such as 
this. 

21. The science supervisor or con- 
sultant should have a greater 
control than principals on 
the strategies used in teach- 
ing science. 



^ A '2 P, .SD 

7.0 l3ol 9.7 47.0 22.0 

1.2 3.4 9.9 33.9 50.4 

3.2 10.6 I 5 o 0 33.4 37.0 

12 o 6 42.1 21.8 11,6 9.9 



Your Classroom 

22, I am comfortable using only those 1,2 9«9 9.4 5l.l 26.0 

pupil activit?..3s which we worked 
through in the insorvice educa- 
tion classes. 



23. I have used the materials of the 14.8 32,2 13,8 29.5 75 

elementary school science pro- 
gram extensively in my class- 
room in recent weeks. 



24. The children accepted the new 
materials readily. 

25. The concepts were too difficult 
for the children to assimilate. 

26. It was possible to evaluate the 
progress of the students. 



40.9 40.7 12.6 2.7 1.2 

2.2 5.1 13.8 47.7 29.5 
11.4 40 »o 27.6 16.0 3.2 



27. I feel that it is important to 29.1 46,7 12,6 85 22 

be able to evaluate the child- 
ren’s progress. 



o 
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28 „ I now feel more confident of mv 

teach science than 
1 did at the beginning of the 
pro ject. 



M A 2 D 
26.6 45.8 13,6 9, 



29# My principal has encouraged and 

supported my endeavors with this 
program. 



29.1 37.8 19o4 6, 



30. 



I have had difficulty in 
ing the materials needed 
science teaching. 



ob tain- 
in my 



23.5 23.5 9,7 30. 



SD 

0 44 

3 5.3 

3 10,6 



